Skip to content

feat: introduce ManagedResult, RunnerResult, and LDAIMetricSummary#1332

Draft
jsonbailey wants to merge 2 commits intojb/aic-2174/server-aifrom
jb/aic-2388/js-managed-result
Draft

feat: introduce ManagedResult, RunnerResult, and LDAIMetricSummary#1332
jsonbailey wants to merge 2 commits intojb/aic-2174/server-aifrom
jb/aic-2388/js-managed-result

Conversation

@jsonbailey
Copy link
Copy Markdown
Contributor

Summary

  • Adds RunnerResult interface (provider-level result: content, metrics, raw?, parsed? — no evaluations)
  • Adds ManagedResult interface (managed-layer result with evaluations: Promise<JudgeResult[]>)
  • Adds LDAIMetricSummary (flat summary: success, usage?, toolCalls?, durationMs?, resumptionToken?)
  • Adds toolCalls? and durationMs? fields to LDAIMetrics
  • TrackedChat.run() replaces/supplements invoke(), returning ManagedResult with metric summary built from tracker
  • Adds createModel() to LDAIClient and LDAIClientImpl as the preferred replacement for createChat()
  • Updates chat-judge example to use createModel() and run()

Test plan

  • All 188 tests pass
  • chat-judge example updated to use new API

🤖 Generated with Claude Code

@jsonbailey jsonbailey force-pushed the jb/aic-2174/server-ai branch from 46ab0a4 to c751ce6 Compare April 28, 2026 23:12
…IC-2388)

Adds RunnerResult (provider-level result type without evaluations), ManagedResult
(managed-layer result with async evaluations promise), and LDAIMetricSummary (flat
metric summary including resumptionToken). Adds toolCalls and durationMs to
LDAIMetrics. TrackedChat.run() replaces invoke() returning ManagedResult with
LDAIMetricSummary built from tracker. Adds createModel() to LDAIClient/LDAIClientImpl
as the preferred replacement for createChat(). Updates chat-judge example.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
@jsonbailey jsonbailey force-pushed the jb/aic-2388/js-managed-result branch from fe6948b to 192315f Compare April 28, 2026 23:14
Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We need to rename the TrackedChat type to ManagedModel naming. We should also introduce the runner interface as defined in the spec and follow it. This will be a breaking change to move to the .run method and we should drop the old invoke method.

… conversation management

- Add `Runner` and `AgentGraphRunner` interfaces in api/providers/Runner.ts.
  Runner.run takes a prompt string + optional output schema and returns a
  RunnerResult. AgentGraphRunner.run takes a string and returns an
  AgentGraphRunnerResult. Re-export both from api/providers/index.ts.
- Add the supporting `GraphMetrics` and `AgentGraphRunnerResult` types to
  api/graph/types.ts so AgentGraphRunner has its result shape on this branch.
- Rename `TrackedChat` -> `ManagedModel` (file + class). The constructor now
  takes a `Runner` instead of an `AIProvider`. The class is stateless: it
  owns no conversation history, and `run(prompt)` forwards the prompt
  directly to the runner. Drop `invoke()`, `_evaluateWithJudges`,
  `appendMessages`, `getMessages`, `getJudges`, `getProvider`, and the
  internal `messages` field.
- Update `LDAIClientImpl.createModel` to construct a `ManagedModel` with a
  `Runner`. The factory still produces a (deprecated) `AIProvider`, so a
  small `runnerFromAIProvider` adapter wraps it: it prepends the AIConfig's
  configured messages to the user prompt to preserve existing
  system-prompt behavior under the stateless contract.
- Mark `createChat` `@deprecated` (now delegates to `createModel`); keep
  `initChat` deprecated. Update the `LDAIClient` interface accordingly.
- Mark the `AIProvider` abstract class `@deprecated` in favor of `Runner`.
- Update `tracked-chat` and `chat-observability` examples to call
  `createModel` + `model.run()` instead of `createChat` + `chat.invoke()`.
- Rewrite the test suite for the stateless ManagedModel: prompt is passed
  through verbatim, no history is retained, ManagedResult is built from the
  RunnerResult plus the tracker's resumption token. Drop the old tests for
  `appendMessages`/`getMessages`/`invoke`.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
jsonbailey added a commit that referenced this pull request May 1, 2026
… (AIC-2388)

Adds RunnerProtocol.test.ts to verify that the Runner and AgentGraphRunner
interfaces can be implemented as plain objects. The Runner, AgentGraphRunner
interfaces, AIProvider deprecation, and providers/index.ts re-exports landed
in the parent PR (#1332).

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant